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Multi-spectral misregistration of Sentinel-2A 
images: analysis and implications for potential 
applications 
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Abstract—This study aims at = analyzing — sub-pixel 
misregistration between multi-spectral images acquired by the 
Multi-Spectral Instrument (MSI) aboard Sentinel-2A remote 
sensing satellite, and exploring its potential for moving target and 
cloud detection. By virtue of its hardware design, MSI’s detectors 
exhibit a parallax angle that leads to sub-pixel shifts that are 
corrected with special pre-processing routines. However, these 
routines do not correct shifts for moving and/or high altitude 
objects. In this letter, we apply a phase correlation approach to 
detect sub-pixel shifts between B2 (blue), B3 (green) and B4 (red) 
Sentinel-2A/MSI images. We show that shifts of more than 1.1 
pixels can be observed for moving targets, such as airplanes and 
clouds, and can be used for cloud detection. We demonstrate that 
the proposed approach can detect clouds that are not identified in 
the built-in cloud mask provided within the Sentinel-2A Level-1C 
(L1C) product. 


Index Terms—cloud detection, multi-spectral misregistration, 
phase correlation, Sentinel-2, sub-pixel 


I. INTRODUCTION 


HE Multi-Spectral Instrument (MSI) aboard Sentinel-2A 

remote sensing satellite acquires images of Earth’s surface 
in thirteen spectral bands [1]. Bands 2 (blue), 3 (green), 4 (red) 
and 8 (near infrared, NIR) are acquired at 10m spatial 
resolution; red edge bands 5, 6, 7, NIR band 8a and short- 
wave infrared (SWIR) bands 11 and 12 are acquired at 20 m 
spatial resolution; and bands 1, 9 and 10, that are designed for 
aerosols retrieval, water vapor retrieval and cirrus detection, 
respectively, are acquired at 60 m spatial resolution. The MSI 
is designed in such a way, that the sensor’s detectors for the 
different spectral bands are displaced from each other. This 
introduces a parallax angle between spectral bands that can 
result in along-track displacements of up to 17 km in the 
Sentinel-2A scene [2]. Corresponding corrections using a 
numerical terrain model are performed to remove these inter- 
band displacements, so the MSI images, acquired in different 
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spectral bands, are co-registered at the sub-pixel level to meet 
the requirement of 0.3 pixels at 99.7% confidence. However, 
these pre-processing routines cannot fully correct 
displacements for high altitude objects, e.g. clouds or fast 
moving objects such as airplanes or cars. Therefore, these 
types of objects appear displaced in images for different 
spectral bands. The magnitude of the displacement varies 
among pairs of bands depending on the inter-band parallax 
angle. For example, at 10 m spatial resolution, the maximum 
displacement is observed for bands 2 (blue) and 4 (red). 
Hence, the multi-spectral displacement can be used as a 
feature to detect clouds or moving objects. This idea has been 
exploited for geostationary satellites, where spectral bands are 
acquired at slightly different times [3], [4], [5], and very high 
and moderate spatial resolution satellites, where a parallax 
angle between pan-chromatic and multi-spectral bands exist 
[6], [7], [8], [9], [10]. 

Various geometric issues for Sentinel-2A/MSI images have 
already been addressed in previous studies including co- 
registration of multi-temporal Sentinel-2A/MSI images [11], 
and co-registration of Sentinel-2A/MSI images with images 
acquired by other sensors, e.g. Operational Land Imager (OLI) 
aboard Landsat-8 satellite [11], [12], [13]. For these studies, 
well-established image registration techniques were exploited 
that allow automatic co-registration of satellite images with 
sub-pixel accuracy. Both feature-based [12] and area-based 
[11] approaches proved effective in improving co-registration 
at sub-pixel level between multi-temporal Sentinel-2A/MSI 
images and multi-source images (Sentinel-2A/MSI and 
Landsat-8/OLI). The European Space Agency (ESA) regularly 
publishes “Data Quality Reports” which provide quantitative 
overviews of Sentinel-2 product performance, including 
geometric aspects. However, aspects related to multi-spectral 
misregistration of Sentinel-2A/MSI images and _ potential 
applications that can be derived from this characteristic of the 
data, have not been documented in the literature yet. 
Therefore, in this letter, we aim to analyze the misregistration 
between Sentinel-2A/MSI images acquired in the visible 
spectrum at 10 m spatial resolution and explore its potential 
use for detection of clouds and moving objects. 


II. METHODOLOGY 


Phase correlation was the primary method used in this study 
for detecting misregistration between multi-spectral Sentinel- 
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2A images at sub-pixel scale [14], [15], [16]. Phase correlation 
belongs to a group of area-based image matching methods, 
and is a well-established technique for estimating translation 
and rotation (through conversion to polar coordinates) 
between images. In particular, it finds correspondence 
between reference and sensed (slave) images by estimating a 
cross-correlation measure in the frequency domain, using 
Fourier transform. This peculiarity of the phase correlation 
technique allows detection of sub-pixel shifts between images. 
Advantages of phase correlation include [15]: high accuracy 
(at sub-pixel scale), error analysis to remove unreliable and 
erroneous shifts, and robustness. The latter refers to the ability 
of phase correlation to effectively detect shifts in multi- 
temporal or multi-spectral images. In this study, we used a 
computationally efficient procedure based on nonlinear 
optimization and Discrete Fourier Transforms (DFTs) to 
detect sub-pixel shifts between reference and sensed images 
proposed by Guizar-Sicairos et al. (2008) [16]. This approach 
was used to co-register Sentinel-2A/MSI and Landsat-8/OLI 
images [11]. For details on the algorithmic implementation of 
phase correlation for sub-pixel image registration, we refer the 
reader to [15], [16]. 

We applied phase correlation on a sliding window basis 
(patches) whereby a subpixel shift between two images is 
detected for each window with the size n,, x ,, and step size n, 
(Fig. 1). Window size should be big enough to contain details 
and features in order to detect the shift between images. At the 
same time, the window size should not be too big, so that the 
shift between images within the window will be uniform. 
Also, window size limits the size of objects that can be 
detected through misregistration of images. Step size with 
which windows are being slid determines the spatial resolution 
of the resulting shift map containing the shift between the 
images. For 10 m images, the window step size of 6 pixels 
(n;=6) will result in a map with an effective spatial resolution 
of 60m, i.e. the shift will be detected every 60 m (without 
additional resampling). 


Fig. 1. A sliding window with the size n,, x n, and step size n, for which a 
phase correlation is run to estimate sub-pixel misregistration. 


The phase correlation approach was applied to multi- 
spectral (B2 — blue, B3 — green and B4 — red) Sentinel-2A 
images at 10 m spatial resolution. The parallax angle between 
odd/even detectors for bands B2, B3 and B4 is 0.022, 0.030 
and 0.034 (all in radiance units), respectively [17]. These 
values along with the detectors’ configuration lead to the 


following intra-band parallax angles: 0.006 radians for B4-B2 
bands, 0.002 radians for B4-B3 bands, and 0.008 radians for 
B3-B2 bands. The corrections, applied during pre-processing, 
will fix multi-spectral shifts, induced by parallax, for static 
and low altitude objects to meet the requirement of 0.3 pixels 
for multi-spectral band registration [2]. It is assumed, 
however, that larger shifts (©0.3 pixels) will be observed for 
moving and high altitude objects that can be detected with a 
sub-pixel phase correlation approach. These larger shifts can 
be separated from shifts for static and low altitude objects, and 
this will constitute a basis for detecting moving objects and 
high altitude objects such as airplanes and clouds. For 
example, an intra-band parallax angle for the B4-B2 pair is 
Adga,po=0.006 radians, and at nominal Sentinel-2A altitude of 
h=786 km and speed of v=7.44 km/s, this angular difference 
will introduce a time-lag [6] of Atgapo=Adpspoh/v=0.63 s. 
Therefore, for static objects at 500m altitude, the pixel 
displacement (in B2 and B4 images) will be 0.006*500=3 m, 
or 0.3 pixels at 10 m resolution; for moving objects, the 
minimum detectable target motion at 0.3 pixels level will be 
~4.7 m/s (17 km/h). 

Finally, no NIR (B8) band was used in this study, because 
of the following reasons: (i) parallax angle for B8 band is 
close to the one of B3 (green), and therefore it does not lead to 
shifts larger than those in visible bands; (ii) significant 
differences in the spectral response of NIR compared to 
visible bands, especially in vegetation, that resulted in the 
phase correlation detecting erroneous shifts in images. 


Ill. DATA DESCRIPTION 


We used the standard Sentinel-2A Level-1C (L1C) product 
which is radiometrically and geometrically corrected with 
ortho-rectification and provides top of atmosphere (TOA) 
reflectance values [2]. Along with quality information, L1C 
also contains a built-in vector cloud mask. The product is 
provided in tiles of 109.8 km x 109.8 km size in the Universal 
Transverse Mercator (UTM) projection with the World 
Geodetic System 1984 (WGS84) datum. Each Sentinel-2 tile 
is referenced through the U.S. Military Grid Reference System 
(MGRS), with a tile identifier consisting of two numbers and 
three letters e.g. 16TCK. The first two numbers in the tile 
identifier correspond to the UTM zone while the remaining 
three letters correspond to the tile location [17]. Three 
Sentinel-2A scenes covering two study areas (Table 1) and 
featuring diverse land cover (Fig. 2) were used in this study to 
investigate multi-spectral misregistration and the potential for 
detecting moving objects and clouds. First, an area in the US 
was selected consisting mainly of agricultural land, and a 
second region in United Arab Emirates (UAE) including urban 
areas and desert. The US scene was used to explore the 
potential of using multi-temporal images misregistration for 
cloud detection and the UAE scene was used to detect the 
moving objects. 
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TABLEI 
DESCRIPTION OF DATA USED IN THE STUDY 


Region/Country a Date of acquisition 
Bondville, IL, US 16TCK 15 June 2016 and 21 May 2017 
Dubai, United Arab 40RCN 13 June 2017 


Emirates (UAE) 


88°30W 


25°0'N 


55°30E 
{b) 


Fig. 2. Examples of TOA true color images (combination of bands 4, 3, 2) 
acquired by Sentinel-2A/MSI over study areas: Bondville, IL, US on 15 June 
2016 (a) and Dubai, UAE on 13 June 2017 (b). 


IV. RESULTS 


Table 2 shows the average absolute shifts between different 
combinations of bands B2, B3 and B4 for subsets from the 15 
June 2016 and 13 June 2017 Sentinel-2A scenes, without the 
presence of clouds and/or moving objects. The estimated shifts 
are within the requirement of 0.3 pixels and conform to results 
published in ESA “Data Quality Report” [18]. Our results are 
also in agreement with the relative parallax angles between 
bands B2, B3 and B4, as the largest shift (in terms of average 
and standard deviation) is observed between bands B4 and B2 
(parallax angles are 0.034 and 0.022 radians), and smallest 
shift is observed between bands B4 and B3 (parallax angles 
are 0.030 and 0.034 radians). 

The magnitude of shifts between visible spectral bands in 
Sentinel-2A images increases, when moving or high altitude 
objects are present. Fig. 3 shows a Sentinel-2A true color 
image as well as shift maps for pairs of B2, B3, and B4 bands. 
A moving airplane can be seen with a “rainbow” effect due to 
shifts between multi-spectral bands, and these are observed in 
shift maps calculated using a phase correlation approach with 
a sliding window (n,=16, size n,=2). Misregistration shifts 


between multi-spectral images for a moving airplane are 0.5 to 
1.1 pixels depending on the bands (Table 3). However, shifts 
for standing airplanes (shown in bottom left part of Fig. 3(a)) 
are within the ESA requirement and less than 0.3 pixels (Fig. 3 
(b, c, d)). 


TABLE II 
AVERAGE SHIFTS (IN PIXELS AT 10 M) WITH STANDARD DEVIATION BETWEEN 
MULTI-SPECTRAL SENTINEL-2A IMAGES 


Scene ae of Band : Shift, pixels 
points combination 
Dubai, 13 June 2017 154449 B3-B2 0.037+0.028 
B4-B3 0.033+0.025 
B4-B2 0.043+0.089 
US, 15 June 2016 104329 B3-B2 0.049+0.057 
B4-B3 0.051+0.053 
B4-B2 0.041+0.063 
TABLE III 


AVERAGE SHIFTS (IN PIXELS AT 10 M) WITH STANDARD DEVIATION BETWEEN 
MULTI-SPECTRAL SENTINEL-2A IMAGES FOR A MOVING AIRPLANE ON A 
DUBAI SCENE ACQUIRED ON 13 JUNE 2017. SHIFTS WERE AVERAGED OVER A 
5X5 WINDOW CENTER OVER THE MOVING AIRPLANE. 


Band combination Shift, pixels 


B3-B2 0.652+0.306 
B4-B3 0.500+0.142 
B4-B2 1.104+0.645 


106°3824°W 


106°3824°W 108°37'48°W 


Shit, pixels 
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Shift, pivets 
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Shit, pixels 


Ho: 


(a) 


Fig. 3. A subset of Sentinel-2A true color image (combination of bands B4, 
B3, and B2) acquired on 13 June 2017 (a). Shift maps were estimated from 
different pairs of visible bands at 10 m spatial resolution using a phase 
correlation approach with window size n,=16 and step size n,=2: bands 3 and 
2 (b); bands 4 and 3 (c); and bands 4 and 2 (d). 
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Fig. 4. Example of cloud detection for Sentinel-2A/MSI images acquired over the US (tile 16TCK) on 15 June 2016 (a) and 21 May 2017 (d). True color 
images (combination of bands 4, 3 and 2) at 10 m spatial resolution along with the built-in cloud mask (in red) are shown in subplots (a) and (d); shifts 
estimated from band 4 and 2 images using phase correlation and a sliding window with parameters n,,=64 and n,=6 are shown in (b) and (e); cloud masks (in 
magenta) derived from the multi-spectral misregistration using a threshold of 0.2 pixels for shifts are shown in subplots (c) and (f). 


It should be however noted that successful detection of 
moving targets depends on the window size; i.e. if shifts are at 
the order of the window size, the detection of the shifts within 
the window by phase correlation is becoming problematic. In 
that case, the window size should be increased. 

Multi-spectral misregistration of the similar magnitude is 
also observed for clouds on Sentinel-2A images which may be 
used as one of the criteria for detecting clouds. Fig. 4 shows 
subsets of Sentinel-2A images acquired over the US (tile 
16TCK) on 15 June 2016 and 21 May 2017 with the internal 
cloud mask provided for the L1C product, the detected shifts 
between bands 4 and 2 using the phase correlation approach 
with a window size n,=64 and step size n,=6, and the derived 
potential cloud mask from the shifts using a threshold of 0.3 
pixels. 

Visual inspection of Fig. 4 shows that the built-in cloud 
mask of the LIC product does not always detect clouds, while 
the mask derived from the multi-spectral misregistration 
detects most of clouds. It is worth mentioning that the 
proposed approach is not intended to be used as a stand-alone 
criterion for cloud detection, but should be rather combined 
with already established criteria suggested, for example in [19] 
and [20]. The main strength of the presented approach is in its 
ability to detect small and thin clouds that lead to sub-pixel 
misregistration in visible bands of Sentinel-2A images. A 
limitation of the proposed approach is related to difficulties to 
detect clouds with high reflectance values and with a size 
substantially larger than the sliding window size. In this case, 
the window for which the phase correlation is applied will not 
have enough detail to detect any sub-pixel shifts between 


images. However, existing algorithms for cloud detection can 
handle such cases. There can be also a potential situation, 
when the effect of cloud motion and high altitude cancel each 
other out, so that no pixel displacement will be observed. This 
could happen when, the object will be strictly moving along 
the satellite track in the North/South direction. However, pixel 
displacements, when the object is moving in other directions 
will not be compensated by stereoscopic (altitude) parallax. 
One approach to mitigate and reduce the likelihood of this 
situation for Sentinel-2 will be to take advantage of different 
parallax angles between multiple bands (B2, B3, B4), which 
will induce different combinations of objects height and 
velocity values that compensate pixel displacement, and 
estimate the maximum pixel displacement between three band 
combinations (B2-B4, B2-B3, B3-B4). 


V. CONCLUSIONS 


In this letter, we demonstrate a concept for exploiting multi- 
spectral misregistration in Sentinel-2A/MSI images at 10 m 
spatial resolution to detect moving targets and clouds. Because 
of a parallax angle between detectors in the MSI instrument, 
misregistration occurs in multi-spectral images acquired by 
Sentinel-2A. These shifts are corrected with special pre- 
processing routines using a numerical terrain model; however, 
these routines cannot correct these shifts for moving or high 
altitude objects. Although this characteristic of the data might 
be considered as a disadvantageous artifact, in this letter we 
have shown that it can be beneficial for some applications. 
Moving and high altitude objects can be shifted more than 1 
pixel at 10 m resolution for Sentinel-2A visible spectral bands, 
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and these shifts can be detected using a phase correlation 
approach at sub-pixel scale. We have demonstrated that this 
Sentinel-2A/MSI feature of multi-spectral misregistration has 
the potential for detecting moving targets, e.g. airplanes, and 
clouds. Further work should be directed to combining this 
approach with other criteria to make this approach applicable 
in the operational context. 
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